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The undersigned hereby declare that: 

1. I, John Dolloff, am the inventor of the invention claimed in U.S. Patent Application 
No. 10/038,644, entitled "Replacement Sensor Model for Optimal Image Exploitation," filed 
January 08, 2002 ("the '644 application"). 

2. The '644 application discloses a sensor model for describing a ground-to-image 
relationship for image sensors. More specifically, the present invention relates to a replacement 
sensor model that is generic to a plurality of image sensors and that reliably provides a full range of 
image exploitation. 
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3. I have read and understand the Office Action dated February 28, 2006. I understand 
that Dial, Jr. et al. (US 6,735,348) was filed on May 1, 2001 (the "Critical Date"), that this reference 
has been cited as prior art against the pending claims 1-6, 12-13 and 23, and that the Examiner 
considers the pending claims listed prior to be anticipated by the Dial reference. 

4. I also understand that claims 7-11 and 14-16 have been objected to as being 
dependent upon a rejected base claim, but would be allowable if rewritten in independent form 
including all of the limitations of the based claim and any intervening claims. 

5. The invention as claimed in the '644 application was reduced to practice prior to the 
Critical Date. To establish the date of completion of the invention of the '644 application prior to 
the Critical Date, the following enclosed documentation is submitted as evidence: "An Evaluation of 
Rational Functions for Photogrammetric Restitution," ISPRS Amsterdam 2000 Conference, July 16- 
23,2000 ("Exhibit A"). 

6. As shown in Exhibit A, the method as claimed in the present invention was created 
prior to the Critical Date. The published paper in Exhibit A discusses the method of the present 
invention including generating an error covariance, as well as other elements of the present invention 
(see e.g., Section 5, page 261). 

7. The published paper in Exhibit A bears a date that is prior to the Critical Date (i.e., 
"2000"). 

8. We hereby declare that all statements made herein of our own knowledge are true and 
that all statements made on information and belief are believed to be true; and further that these 
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statements were made with the knowledge that willful false statements and the like so made are 
punishable by fine or imprisonment, or both, under Section 1 001 of Title 1 8 of the United States 
Code and that such willful false statements may jeopardize the validity of the application or any 
patent issued thereon. 
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ABSTRACT 

This paper addresses the questions of whether rational functions are suitable for photogrammetric restitution. With the 
increasing number, and greater complexity of sensors becoming available, and the need for standards in transferring 
orientation data, the use of polynomials offers an attractive solution. However there are concerns about the accuracy 
and robustness of such methods, and the paper sets out to examine these concerns and also to propose a method of 
removing some of the shortcomings of current methods. The paper reviews the current information available on 
rational functions and sets out the methods being used and reports on results which are available. A new method of 
dealing with error propagation is proposed. The paper concludes that the use of polynomial functions can offer an 
effective method of photogrammetric restitution for a wide range of sensors but that the models available have not been 
tested or used sufficiently to be certain that error or instability will not be a problem. There does however seem to be 
no case to use polynomial functions with standard frame cameras, as existing methods are rigorous and proven to work, 
unless the most important requirement of the user is having only one sensor model for all sensors. 

1. INTRODUCTION 

"Non rigorous" transformations between object space and image space have been used to a limited extent for a 
number of years. The best known example is probably the Direct Linear Transformation (DLT), used mainly for close 
range photogrammetry. Other methods which have come into use more recently are grid interpolation methods and 
polynomials, or ratios of polynomials (rational functions, RFs). These have been used to replace more rigorous sensor 
models for satellite data where there is a confidentiality aspect, a need for generality or where greater speed is needed. 

Rational functions in particular are being used to define the image geometry model of high resolution satellite data, 
allowing the rigorous sensor model to remain confidential. Using rational functions provides a method which can be 
used for any sensor model, hence possibly defining a universal transfer standard; it also provides confidentiality and 
efficiency in computation, especially in the real time loop of photogrammetric software. There are, however, serious 
concerns: rational function models are not "rigorous" and the accuracy inherent in the original rigorous sensor model 
may be lost in using RFs; because of this, assessment of precision cannot easily be made and accuracy assessment 
must be made through the use of check points; since the coefficients have no physical interpretation, it is difficult to 
determine necessary changes in these coefficients in order to reduce errors on check points; undetectable interpolation 
errors may be introduced between check points; determination of suitable defining functions and of the required 
number of points is complex; solutions using rational functions can be unstable. The lack of error propagation 
information accompanying the RF causes problems for accuracy assessment and optimal solutions when performing 
geopositioning using the imagery and associated RF's. Error propagation information quantifies the support data 
errors associated with the original, rigorous model as well as the fit errors, although the latter should be negligible 
relative to the former. 

The paper will first explain what rational functions are and how they are used for photogrammetric applications. A 
review of literature on the subject will be presented and discussed with a view to answering a number of basic 
questions relating to the accuracy and stability of such methods. The procedures for computing rational functions and 
the choice of parameters will be analysed and weaknesses identified. An ISPRS working group has been set up to 
consider these questions under the chairmanship of the first author and members of that working group are engaged in 
testing various aspects of rational functions. The results of their work are included in the paper. The problem of error 
propogation is considered in detail and further results with this method are presented. 
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BACKGROUND TO RATIONAL FUNCTIONS 



The traditional photogrammetric method of transforming from object space to image space is using the collinearity 
equations. The generic term for the model used is an image geometry modelS. The method is highly suited to frame 
cameras and non-linear effects caused by lens distortion, film distortion or atmospheric effects (for example) are dealt 
with by additional parameters or by corrections after the linear transformation. The introduction of push broom 
sensors such as SPOT cannot be so easily dealt with by the collinearity equations as each line on the image has 
different elements of exterior orientation. These have been dealt with by either using iterative methods (Gugan and 
Dowman 1988) or by generating a three dimensional grid of 'anchor points' and interpolating between these (Konecny 
et al, 1987). Kratky (1989) described how polynomials could be used to speed up computations within the real time 
loop of a photogrammetric plotter. After the parameters of the image geometry model have been rigorously 
determined they can be used to define the required transformation for a grid covering an image for 3 levels of 
elevation extended over the full range of the ground scene. Following that a suitable polynomial function is fitted to 
the spatial grid through a least squares solution, so defining parameters which can be applied in real time conversions 
with enormous time savings. The term replacement sensor model is used in this paper as a generic term for such 
models. 

In military environments an additional requirement has arisen: that is the use of many, often complex, sensors which 
are not of the frame camera type. This could also become an issue for civilian use as more sensors become available. 
A different image geometry model is needed for each sensor and this causes complications in implementation. A 
further difficulty arises when data from sensors whose geometry is to be kept confidential is to be released to a wider 
body of users. 

It should be noted in passing that the new generation of aerial sensors, line scanning and mosaiced arrays may cause 
similar problem for commercial exploitation, however this mass market will create a need for pre-processed data 
which will be made available to users in a resampled format. 

A number of generic techniques are available to transform from 3D object space to image space. The first approach is 
the polynomial, or warping approach. This is unsatisfactory for photogrammetric use and will not be discussed 
further. 

The second approach in the grid interpolation model which is basically the anchor point method in which the image 
co-ordinates for a 3D grid in object space are computed rigorously so that the image co-ordinates of other points can 
be found from interpolation. The main disadvantage of this method is the large number of grid points required for 
accurate interpolation and hence a large volume of stored data. The method is however generic and can be used for all 
types of imagery and is independent of the type of imagery. 

The third approach is rational functions which is widely used in the US intelligence community (OGC, 1999). An 
image co-ordinate is determined from a ratio of two polynomial functions. 



= PiU n ,y n ,z n ) 



c = P2(x„,y n ,z„) 



where : 

r n = normalised row index of pixel in image 
c n = normalised column index of pixel in image 
x n ,y n ,z n = normalised ground coordinate values 



The polynomials are computed by fitting the rational functions to image grids and object grids set up at different 
elevations using the rigorous image geometry model and least squares adjustment. 



'image geometry model is the term used by the Open GIS Consortium (OGC). Alternative terns are rigorous sensor 
model, image sensor model or simply sensor model. 
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Rational functions are used within the defence organisations and commercial mapping agencies in the USA and 
elsewhere and are already available within a number of digital photogrammetric workstations such as ZI Imaging 
Image Station (Madani, 1999) and LH Systems SOCET Set. 

Madani (1999) gives the following advantages: 

sensor independence; 

• sufficient speed for real time implementation; 

• Support of any co-ordinate system; 

easy inclusion of external sensors into work flows via image object grids used to generate upward projection; 

• high accuracy since RFs resemble projective equations; 
ability for one set of RFs to be used for all types of imagery. 

Medani also give the following disadvantages: 

the method cannot model local distortions (CCD array distortions); 

• the terms have no physical meaning, making their interpretation and importance difficult to ascertain; 
potential failure due to zero denominator; 

• possibility of correlation between terms of the polynomials; 

• accuracy decreases if an image is too big or has too high frequency of image distortion. 

However to counter these disadvantages checks are built in to avoid zero crossing and to check for the determinability 
and significance of each RF term. Parameters that show high correlation will be eliminated from the functions. 

OGC (1999) also notes a number of disadvantages which include limited accuracy in fitting to the rigorous image 
geometry model and a complex non-linear fitting process to avoid the denominator function becoming zero. In order 
to overcome these problems a fourth method called the 'the universal real-time image geometry model', which will be 
referred to as a Universal Sensor Model (USM), has been developed and described in the Open GIS Consortium 
(OGC) Abstract Specification (OGC, 1999). The USM has been accepted as the new NIMA standard real-time 
geometry model. This was developed and first implemented by GDE Systems, Inc (now BAE SYSTEMS). 

The USM has the following objectives: 

• to support exploitation of images collected by all current and future imaging sensors; 
to provide very high fitting accuracy: 

• to allow implementation in any image exploitation system. 

The USM is based on RFs but offers higher storage efficiency than other RFs at the cost of some additional 
complexity. It is a dynamic RF polynomial, in that it allows for the selection of a minimum number of RF polynomial 
coefficients to meet a desired fit accuracy relative to the rigorous sensor model of complex sensors. The coefficients 
(support data parameters) are produced by fitting the model to any rigorous sensor model, typically following 
completion of triangulation or other adjustment. This post-triangulation fitting scenario is desirable so that the 
rigorous model which the RF is fitted to is as accurate as possible, and that additional USM support data, such as 
accuracy assessment information, can be fully populated. The support data format of the USM also allows for tables 
of high order corrections that can be (optionally) interpolated and applied to the RF evaluation. For most sensors, they 
are not generated or needed. 

The USM extends the rational function model in the following respects: 

one image geometry model can be divided into multiple subdivisions or sections; 

• the modified polynomials are of variable order, up to the fifth power of the two horizontal co-ordinates and up to 
third power for elevation; 

the denominator polynomials can be omitted, and usually are omitted; 

• image positional correction tables can be incorporated; 

• the ground co-ordinates can be in any defined ground co-ordinate system. 

The BAE fitting software has the following characteristics: 

1 . it is a fully automatic process other than available operator review; 

2. only the rational polynomial's numerator is fitted; 
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3. the polynomial's independent variables are normalized geographic coordinates; 

4. least squares fitting of the polynomial coefficients is performed using an image measurement-ground point 
correspondence grid consisting of 100 points over the image at each of 5 elevation planes (500 points total); 

5. assessment of the fit accuracy is performed by comparing the ground-to-image mapping of the original, rigorous 
sensor model to the polynomial at a grid consisting of 400 points over the image at each of 10 elevation planes 
(4000 points total); 

6. if a desired 90% fit error tolerance is met (90% of sampled fit errors are less than the tolerance), the fit is 
complete, otherwise the polynomial order is increased and the process reported; 

7. if the fit tolerance is still not met, the order is increased to a maximum, and if that doesn't work the image is 
segmented and the entire fit process started again. If for example, the image were segmented into two pieces, two 
polynomials would be generated and supplied to the user with a method to determine which polynomial to use 
based on a linear polynomial that covers the entire original image. 

The above fitting process eliminates the need for external checkpoints to assess interpolation error. The role of 
external checkpoints becomes accuracy assessment of the rigorous sensor model, when required. 

In addition to generating the polynomial, BAE software also populates ancillary (optional) USM support data when 
possible. For a given image, this data includes a ground space absolute and relative accuracy estimate for an assumed 
monoscopic extraction with an identified elevation source accuracy, and for an assumed stereoscopic extraction using 
this image and a specified stereo mate. Contributing errors are sensor support data errors, and all accuracy estimates 
are based on appropriate error (covariance) propagation.. In addition, 90% (0.9 probability) polynomial fit error 
(pixels) is included as a separate item. Y-parallax statistics can also be included, along with shear statistics relative to 
other overlapping images or stereo models. 

3. DISCUSSION 

The arguments for the use of rational functions can be summarised as follows: 

Universality - the method can be used to transform image co-ordinates from an image produced by any imaging 
sensor into object space co-ordinates; 

coefficients can implicitly contain all kinds of effects (sensor geometry, Earth curvature, refraction, self 
calibration parameters) 

partial images can be processed; 

the user need only obtain and maintain one sensor model that covers many different sensors. 

Confidentiality - they allow image data to be used to create object co-ordinates without knowledge of the sensor 
model, the military already make use of this and data from the new high resolution satellites will probably be 
processed using USMs; 

Efficiency - when used in the real time loop of photogrammetric plotters, fewer operations are required 
Transfer - they make transfer of image orientation data easy. 

The following disadvantages may apply: 

Loss of accuracy - the method is not rigorous and can hence lead to errors. [BAE SYSTEMS believes that the USM 
can produce sufficient accuracy when fitted to any rigorous image geometry model using well designed 
software]. 

Instability - USMs are mathematical approximations of the physical world. Therefore, the coefficients do not have a 
physical interpretation, and thus the models can mask systematic errors and introduce interpolation errors. In 
addition care has to be taken to avoid over-parameterisation and numerical instability of the solution. This is 
especially important in the case of RF, since in this case there is a denominator which might go to zero without 
warning. But also for polynomials over-parameterisation can lead to very strong correlation between the 
coefficients to be determined, and thus to numerical instabilities. 

For frame imagery RFs are similar to Direct Linear Transformation (DLT), a method which is known to be 
numerical unstable for aerial imagery. The reason is an over-parameterisation of the solution. 
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Failure - for airborne raw 3-line imagery an interpolation of the flight path is highly problematic because of the 
sometimes very rough turbulence during flight. Thus, RF will have a limited use only, and one needs to use 
resampled imagery. 

Uncertainly - accuracy assessment is limited to using check points. Errors are not related to physical perturbations so 
that there is no clear indication on what to change if, for example, the errors at the check points are too high; 
lack of complete and rigorous error propagation information. 

Complexity - in defining function and number of points required, possibly leading to instability and further loss of 
accuracy; 

Not the best method for frame images. 

There is no unanimity amongst the vendors which have been contacted on the role of RFs. The consensus seems to be 
that rigorous models are preferred but RFs are inevitable for some sensors. There is no strong feeling that RFs should 
be adopted as a universal transfer standard. 

The information gathered so far indicates that there is a role for RFs but not that they should be the only standard. 
Amongst photogrammetrists there is a very strong feeling that there is a need for a standard which incorporates a 
rigorous sensor model. 

Key RF performance questions requiring further research and testing are: 

• How robust are they? 

How accurate are they? What are the sources of error? 

Once a USM is defined for a particular sensor, can this be used for all data from that sensor? What happens if the 
sensor is unstable (e.g. IRS), can the parameters be changed and the model still be robust? Are USMs for a sensor 
model independent of terrain? 

Due to the many variables and their interactions affecting RF performance, including sensor characteristics, imaging 
geometry, image size, terrain elevation range, and the validity of the rigorous sensor's mathematical model, it is 
unrealistic that a theoretical mathematical analysis can answer many of the above questions. Empirical testing is 
required. (The following section documents some of the testing performed to date.) However, the following 
observations can be made: 

1. Although RFs with denominators can be unstable, a numerator only USM should be stable assuming the fit 
procedure is adequate, i.e., enough redundancy of fit points, enough fit planes covering the possible range of 
elevation, and fit points that span (or exceed) the image (segment). However, unstable results are still possible if 
the polynomial is evaluated outside its valid ground space domain. 

2. The general sources of RF error are fitting error relative to the rigorous model, and of course, any error in the 
rigorous model itself, primarily modeling error and error in the sensor support data. Fitting errors are generally 
due to an inadequate polynomial representation (terms selected), interpolation error (evaluation at points other 
than those used in the fitting grid), and an inadequate fitting procedure. 

3. In general, it's not feasible to predefine an RF (i.e., terms or coefficients used) that will work for any image from 
a given sensor other than for some sensors with benign imaging geometry and reasonably invariant image size. 
However, a fitting process can be predefined like BAE's USM that should work for all images from almost any 
sensor. However, for a desired fit accuracy, the resultant polynomial order and possibly number of image 
segments can vary. 

There are other issues which are raised but which can probably only be answered from experience of use with RFs: 

Would they be accepted by the user community if they are too complex? What might be acceptable to NIMA 
might not be acceptable to the average DPW/IPS user. 

Could a basic set of physical systematic errors, which should be corrected before employing USMs, be defined? 

4. TEST RESULTS 

Kratky (1989) used polynomials to carry out the transformation from object co-ordinates to image co-ordinates of 
SPOT data in the real time loop of an analytical plotter. The transformation can be rigorously determined in advance 
for a 5 x 5 grid for three levels of elevation, covering the full elevation range of the ground scene. A suitable 
polynomial function is derived to fit the spatial grid through a least squares solution, so defining parameters which can 
be applied in real time conversions. Kratky states that 'Absolute errors of the transformation are within IdZI < 0.2m 
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for the range Ah = lm and ldZI<0.5m for Ah = 4 km, both cases being computed for the maximum side view angle 27° 
of the imaging sensor." Baltsavias and Stallmann (1992) tested the geometric accuracy of polynomial mapping 
functions (PMFs) with a SPOT stereo pair. 136 points were used with a height range of 350-2100m. When using 
different versions of Kratky's SPOT model the accuracy in X, Y or Z on control points varied from 1.0m to 4.7m and 
from 6.3m to 16.0m on the check points. In comparing the computed co-ordinates of 136 points by a rigorous 
solution and with polynomial mapping functions, the difference did not exceed lm in object and lum in image space, 
thus verifying Kratky's results. Baltsavias and Stallmann also show that these functions can be used in image 
matching and orthoimage generation and that a good approximation of the epipolar line can be derived using PMFs 

Madani (1999) reports on the use of rational functions with the orientation of 12 SPOT scenes. The model was set up 
using a rigorous analytical triangulation adjustment with 14 control points and 12 check points. This gave RMS 
values on the check points of 8.5m in planimetry and 6.8m in elevation. To quote from Madani (1999): 

"A total of 50 ground (control/pass) points were used in each stereopair for accuracy analysis. Using the image 
co-ordinates of each stereo pair, the corresponding ground co-ordinates were computed via the RFs using least 
squares. The RMS radial differences between the originally given and the computed ground co-ordinates of 
identical points was 0.18m and the RMS image parallax was about 0.1 5um. The maximum radial difference was 
0.53m and the maximum parallax was about 0.2um. these results show that the RFs expressed these SPOT scenes 
very well. 

"In the second part of the RF accuracy analysis, DTM points were automatically generated for these two stereo 
pairs. Z values of about 50 points per stereo pair were interpolated from the corresponding DTM data. The RMS 
Z-difference, between given/computed and interpolated points, for the first pair was 1 1 .5m and for the second pair 
9.8m. Again these results show that properly selected rational function co-efficients, can be used in the real time 
loop of digital photogrammetric systems." 

Ramon Alamus (2000) from the Cartographic Institute of Catalonia (ICC) has tested the use of rational functions with 
MOMS data. This was in mode A, (2 stereo plus a high resolution channel), acquired during the MOMS-02-D2 
mission on the shuttle in 1993. The scene covers an area of 120km x 40km over the Andes between Chile and Bolivia. 

About 50 ground points were identified from Aerial photography. A strict model of the camera and trajectory were 
adjusted using the GeoTeX-ACX ICC software. 

The mathematical model took into account position and attitude of the spaceborne platform (the Shuttle). On the 
images there were over 1000 tie points (courtesy of Institut fur Optoelectronic - DLR, Oberpfaffenhofen). In the 
adjustment only 4 ground points of the 50 identified there were used. The rest were used as check points. Results 
achieved using these data are: 8m rms error in planimetry and 17m rmse in height. 

The rigorous model has been interpolated by rational functions using Intergraph software developed for ICC. These 
RFs have been used to stereoplot a MOMS pair (Channels ST6 and ST7, the low resolution channels pixel size of 13.5 
m). A set of 290 points uniformly distributed on the scene were selected and measured as part of the aerial 
triangulation procedure using the Intergraph stereoplotter (ISSD) and then these were remeasured on the ground. The 
co-ordinates of the manually identified points were checked against the adjusted co-ordinates of those points. 

The result are 286 points were gave rms errors of 7.9m in X, 8.1m in Y and 12.8m in Z. These errors are similar in 
planimetry to those achieved in the fit to the rigorous model, and better than the fit in height. 

The results from Madani and Alamus indicate that the rational functions available in the Intergraph software can be 
used without loss of accuracy. Neither report makes any mention of stability of the solution. 

Papapanagiotou (2000), from the University of the Aegean has developed a general model for simulating stereo-image 
geometry using polynomials. The polynomials have some similarities with the USM, for example the denominator 
polynomial is always omitted, the polynomials are of variable order and the ground co-ordinates can be in any defined 
ground co-ordinate system. There are also some aspects of the model that are totally different: 

1. The terms of the polynomials don't have to be the ground co-ordinate values. The image co-ordinates of one 
image are used and the ground elevation. This is similar to Kratky's polynomial mapping functions (Kratky, 
1989). Other combinations are also possible for example, three out of the four image co-ordinates. 

2. A rigorous model is not used to obtain control and check point data. The polynomials are constructed purely on 
the control and check data available from other sources e.g. maps. This is to say, no rigorous model has to be at 
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hand, but the cost is additional ground data - usually more that 25 points. Using a rigorous model constrains the 
polynomial accuracy to that offered by the model used. 

3. The order and form of the polynomials is automatically computed for each stereo pair under study. This is 
achieved by a trial and error method based on check point RMSE which completes in a few seconds. This method 
has some similarities with the Group Method of Data Handing (GMDH) used in the so called "polynomial neural 
networks". 

Papapanagiotou,'s test results indicate that similar, if not better, accuracy can be obtained using the above method. The 
model has been tested with two stereo pairs one SPOT and one MOMS-2P. For the SPOT stereo pair the same RMSE 
for the check points was obtained as with Kratky's strict model in X and Y. The height (Z) RMSE was by lm less. For 
the MOMS stereo pair the RMSE obtained was better than Kratky's model in almost all cases (linear and quadratic). 
The method was also tested on two small parts of a second stereo pair. The total RMSE obtained for the check points 
was 4.8m and 5.5m, respectively. Another indication of the great flexibility of the proposed model (and polynomial 
models in general) is that the results obtained for a pair of aerial photographs scanned by an ordinary A4 DTP scanner 
without any prior calibration was similar to the maximum accuracy provided by a map sheet of scale 1:5,000 used to 
extract ground data. 

Papapanagiotou does not argue that polynomials can replace rigorous models and agrees about problems of instability 
and extrapolation errors, but he believes that they don't lack in accuracy. Polynomials do not have to be complex and 
are simple and easily understandable by non-experts. Another conclusion was that if tests are made with controlled 
erroneous data, the accuracy deteriorates rather quickly but in all cases the erroneous points can be identified and 
corrected or ignored. 

As indicated above BAE (Whiteside, 1999) firmly believes that the Universal Sensor Model (USM) is as accurate as 
anyone might want, when fitted to any rigorous image geometry model using well designed software, such as existing 
BAE software. The USM has been tested with a wide variety of images and image types, and the maximum fitting 
error has never been greater than about 0.25 pixel spacing. Typical results are shown in Table 1 (Craig, unpublished 
data, 1997). 



Sensor 


Image size 
(pixels) 


Ground sample 
distance 


Polynomial 
order 

(x-y-z) 


90% fit errors 
(pixels) 


SPOT lAPan 


6000 x 6000 


10m 


2-2-1 


0.05 


Landsat TM 


7000 x 6000 


30m 


2-2-1 


0.01 


Commercial 
frame camera 


18000x 18000 


0.3m 


4-4-2 


0.2 


RADARSAT fine 
beam 


9000 x 8000 


6m 


3-3-2 


0.06 



Table 1. Results of testing the USM at BAE SYSTEMS 



Neither image segmentation nor correction tables are needed. The 90% fit error is defined as the 90% level of fit error 
samples, the image pixel difference between the rigorous model and polynomial evaluated at common ground points. 
For each sensor, the resultant 90% fit error as well as the polynomial order was a function of the 90% fit tolerance. It 
was set outomatically to the smaller 0.25 pixels and (lm/GSD) pixels. Other factors influenced the polynomial order 
as well. In particular, the commercial frame camera's wide field of view, relative to the space sensor's was a major 
contributor to the high polynomial order. In addition, the USM format allows for (but does not require populating) all 
cross terms (see equation [3]) where other RFs do not. That is, even though the SPOT fit used only order 2 in x, order 
2 in y, and order 1 in z, there are terms that have a combined order of up to 2+2+1=5. 

i=0 ;=0 *=0 i=0 j=0 k=0 

These tests show that the accuracy possible with RFs is quite suitable for most applications. They do not show any 
disadvantages although information is not available on how exhaustive the tests were. In other words, the tests do not 
demonstrate that the disadvantages claimed are not well founded. 
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5. IMPROVED ERROR PROPAGATION 

5.1 Background 

As mentioned earlier, error propagation information is extremely limited in current replacement sensor models, i.e., 
polynomials and rational polynomials fit to the original rigorous sensor model's ground-to-image relationship. In 
particular, for a given image, the USM's error propagation consists of a summary of expected ground point extraction 
accuracy assuming a monoscopic extraction or a stereoscopic extraction based on the image and a specified stereo 
mate. Thus, USM error propagation information is limited to a ground space based summary over the area within an 
image footprint. It does not provide image space based error propagation information required when performing 
extractions using this image with additional images besides the specified stereo mate. 

Rigorous and complete image space based error propagation information is critical for optimal geopositioning and 
reliable error propagation. It allows for minimum variance ground point solutions (extractions) and for reliable 
absolute and relative accuracy predictions accompanying those solutions. The original rigorous sensor model 
provides the required image space based error propagation information via the sensor support data error covariance 
(when available) for that image and any other images with correlated errors. Correlations can be introduced either by 
imaging on the same pass or by subsequent triangulation of the sensor support data, 

A new method has been developed by one of the authors (Dolloff) to provide image space based error propagation 
information to accompany the replacement sensor model's (rational) polynomial. This information and the algorithm 
to generate it are summarized below. Initial testing results are also summarized. They indicate the method's promise, 
although further analyses and testing are required and are on going. 

5.2 Description of the method 

In the new method, the error propagation information for a replacement sensor model is captured in an error 
covariance matrix C A associated with a pre-defined vector of adjustable parameters A of the polynomial. This error 
covariance is generated at the same time the polynomial is generated. The generation process requires the original 
rigorous sensor model and C s , the a priori error covariance of the model's support data parameters S (e.g., sensor 
position, attitude, etc.). 

Prior to providing specifics of the above process, a representative geopositioning solution technique is first presented 
in order to understand the role of the original rigorous sensor model and its support data error covariance. The 
following is a simultaneous Best Linear Unbiased Estimate for multiple 3-dimensional ground points using conjugate 
image measurements of the ground points from multiple images: 



The various vectors and matrices are defined as follows: 

X - vector of 3-dimensional ground coordinates of multiple ground points 
P - a posteriori error covariance of solution vector X 
M - image measurement vector 
R - mensuration error covariance 

B - partial derivatives of measurements M with respect to X (subscript x) or with respect to S (subscript s), the sensor 
support data parameters for all images 
C s - a priori error covariance of S 

The subscript "o" on X, P, and M corresponds to a priori (initial) values. In particular, the a priori M is generated 
using the rigorous sensor model's ground-to-image transformation. The weight matrix W provides the mechanism to 
weight the various components of M in a manner inversely proportional to the combined measurement uncertainty due 
to mensuration errors and sensor support data errors propagated to image space. The rigorous sensor model's support 




[4] 



setX n to current solution X and redo solution. 
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data a priori error covariance C s is critical for proper weighting, and hence, for an optimal solution X with reliable 
error propagation P. 

The new replacement sensor model error propagation information is based on the concept of an adjustable (rational) 
polynomial. There are two possible variations. 

The polynomial with an image space adjustment is of the following form: 

u = (a0+ a\x + dly + aiz + a4xy +..) + (AaO + Aalx + Aa2y) 
v = (bO + blx + b2y + b3z + bAxy + ..) + (AM) + Ablx + Ab2y) , and 
A T =[AaQ Aal Aa2 AbO Ab\ Abl], where 
AaO, .. , Ab2 are the additional adjustment parameters. 

A total of six low order corrections (adjustment parameters) are assumed and defined as the elements of the adjustment 
vector A. Although the above 6 corrections are typical, any set of low order corrections can be defined. (Note that 
low order corrections can also be similarly applied to a rational polynomial, i.e., the ratio of two polynomials. 
However, corrections are only applied to the numerator.) 

The polynomial with a ground space adjustment is of the following form: 

u = (aO + alx'+a2y'+a3z'+a4x' /+..) 

v = (b0+b\x'+b2y'+b3z'+b4x'y'+..) , where 



x' 




Ax 




1 y -fijx 




y' 




Ay 


+ 


-y 1 a y 


, with 


z' 




Az 




P -a 1 J[z 





[6] 



A T = [Ax Ay Az a p y] 



A six element affine transformation consisting of small ground space corrections to the polynomial's independent 
variables are assumed. These corrections are defined as the elements of the adjustment vector A. Although the above 
6 corrections are typical, other ground space adjustment parameters can be defined. 

For both the image space adjustment and the ground space adjustment, the error covariance is defined as 
EjeA£A T }=C A , where "E" stands for expected value and "E" error. Typically the polynomial is actually 
unadjusted, therefore A=0. However, its value (zero) is in error. This error is to represent the error in the original 
sensor parameters, hence, C A ^ 0. 

C A is generated from C s , the sensor parameter error covariance, in such a manner as to include the effects of all 
intra-image and inter-image sensor parameter uncertainties and their correlations: 

is generated such that B ^ C ^ =B^C^ B J , i.e., the matrix norm 
IjB^C^B^-B^C^iBJllis minimized, where the partial derivatives B A and B s 
are generated over a grid of ground point locations at different elevations and [7] 
within the image footprints. B A is the partial of the image measurements M 
with respect to the adjustment vector A. 

In general, C A is generated fromC s as above for all images with correlated support data errors. If there are m 
images, r sensor parameters per image, and q polynomial adjustable parameters per image, C s is an mr x mr matrix 
and C A an mq x mq matrix. In this case, the adjustment vector A refers to the collection of individual adjustment 
vectors A corresponding to the m images. 
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Given the replacement sensor model's polynomial and C A instead of the original rigorous sensor model andC s , the 
user performs the desired ground point solution with error propagation by simply substituting C A for C s and 
B A forB s when implementing equation [4]. (Note that the polynomial is used to generate the a priori M, and 
that B A is easily computed either analytically or numerically from the polynomial.) Because relationship [7] is 
satisfied, the resultant "replacement" solution and error propagation will be nearly identical to the "original" solution 
and error propagation. Note that when there is more than one block of images with correlated support data errors, the 
actual C A or C s utilized in equation [4], contains the appropriate individual C A or C s correlated blocks down its 
diagonal. 

Operationally, the system that generates the replacement sensor model performs the following sequential steps for each 
image: 

• Generates the ground-to-image (rational) polynomial from the original rigorous sensor model. 

• Defines the components of the adjustment vector A and generates its corresponding error covariance matrix C A 
from the original rigorous sensor model's support data error covariance C s . This step is actually performed in 
common for all images that have correlated sensor support data errors. 

• Generates all appropriate identifiers associated with the above. 

• Outputs the replacement sensor model and its supporting data to the user community. Regarding the adjustment 
vector's error covariance matrix, only the upper triangular portion is required for each group of correlated (same 
pass) images. The particular C A is duplicated (or "pointed" to) in the support data for each of these images. 

The performance of the replacement sensor model with rigorous error propagation ( C A ) has been verified to-date 
with a limited set of real imagery/support data, and a more extensive set of simulated imagery/support data. The 
following results for one simulated scenario are representative. 

5.3 Test results 

A space-born sensor was emulated using a simulated frame camera with seven sensor error parameters consisting of 
position, attitude, and focal length errors. A focal length and vertical ground sample distance of 3 m were assumed. 
Images were 10k x 10k pixels. Six images were simulated, three from each of two passes. The sensor support data 
errors were modeled as time correlated errors for images from the same pass. Figure 1 illustrates the imaging 
geometry for this scenario and figure 2 the corresponding image footprints and horizontal location for the two ground 
points for solution. The elevations above the local tangent plane for the two ground points were 1000 m and 300 m, 
respectively. Table 2 presents the sensor support data error characteristics for images 4-6. Support data error standard 
deviations were ten times larger for images 1-3. 




Figure 1 . Imaging Geometry Figure 2. Image Footprints 
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Error parameter 


Sigma 


Time Const 


Along track 


4 m 


2000 s 


Cross track 


8 m 


3000 s 


Radial 


2 m 


1000 s 


Rotation a 


.00001 rad 


200 s 


Rotation b 


.00001 rad 


100 s 


Rotation c 


.0002 rad 


300 s 


Focal 


.001 m 


5000 s 



Table 2. Sensor Support Data Error Characteristics 

A simultaneous, six image solution for the two arbitrarily located ground points was performed 100 times, with a 
different set of random numbers utilized each time consistent with all error sources, their standard deviations, and their 
correlations. (All error sources were assumed unbiased.) Besides sensor support data errors, mensuration errors (0.5 
pixel one-sigma) and errors in the a priori estimates of the two ground points (1000 m one-sigma) were also included. 
Each solution was actually performed using equation (4) two separate times: (1) the "original solution", using the 
original rigorous sensor model and its error covariance, and (2) the "replacement solution", using the replacement 
sensor model and its error covariance. 

Polynomial fitting error associated with the replacement sensor model were assumed zero - a reasonable 
approximation when using USM that also assures identical random errors affecting both solution techniques. In 
addition, prior to each of the 100 cases, the replacement sensor model's C A was generated consistent with equation 
[7] for use by the replacement solution. 

Table 3 presents the simulation results. The solutions' absolute and relative errors are presented as well as their 
corresponding accuracy estimates (standard deviations, or "sigmas"). (Absolute statistics refer to ground point 1, 
relative statistics to the ground point 1 -ground point 2 pair; units are meters.) The errors were calculated as the root- 
mean-square (rms) error over the 100 cases. The sigmas were computed from the solutions' a posteriori error 
covariances and were virtually invariant over the 100 cases. The solution in the first row corresponds to the original 
solution. Note that the actual errors approach their corresponding sigma's, indicative of a well modeled solution. The 
solution in the second row corresponds to the replacements solution with a C A corresponding to a 6 term image space 
adjustment vector A. The replacement solution is virtually identical to the original solution, both in terms of the actual 
solution and its error propagation. 



Solution 


x-y-z (m) 


abs sigma 
x-y-z (m) 


rel rms 
x-y-z (m) 


rel sigma 
x-y-z (m) 


original 


29 24 26 


30 27 28 


19 18 17 


18 16 16 


replacement 


29 24 26 


30 27 28 


19 18 17 


18 16 16 


original 
eq wt 


55 58 53 




29 27 29 




replacement 
2 adj par 


161 102 133 


25 23 24 


30 30 24 


2 2 2 



Table 3. Solution Performance 



Two other solutions are also presented in rows 3 and 4 of the table. Row 3 corresponds to the original solution 
artificially using equal weights for all image measurements, i.e., C s is the identity matrix. Note the degradation in 
solution accuracy, and of course, the error propagation is not applicable. This solution illustrates the importance of 
proper weighting of the various image measurements using the correct error covariance matrix. Row 4 corresponds 
to the replacement solution using a C A corresponding to only a 2 term image space adjustment vector A. (The two 
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terms were aO and bO corrections.) This solution illustrates the need for C A to be generated relative to a reasonably 
defined adjustment vector A. 

For the described scenario, the new replacement sensor model error propagation information successfully captured all 
error propagation information associated with the original rigorous sensor model. These results are also indicative for 
all other scenarios analyzed to date, varying in the sensor support data error characteristics, image geometry, and 
number of images. The only exception was a simulated vertical frame camera at a low altitude (4000 m) and with a 
wide field of view (100 degrees). AC A generated relative to the baseline image space adjustment polynomial (6 
adjustment parameters) did not perform well. However, a C A generated relative to a ground space adjustment 
polynomial with the baseline 6 adjustment parameters performed very well. It provided for a replacement solution 
virtually identical to the original solution. This level of performance was also true for all the other scenarios when 
using the baseline ground space adjustment polynomial. 

A final comment concerns the possible refinement of the replacement sensor model when augmented with the new 
error propagation information. It should be straight-forward to refine the adjustable polynomial by solving for 
corrections to the polynomial, i.e., the adjustment vector A becomes non-zero. The a priori value of A (zero) has an a 
priori error covariance C A . This refinement process could involve other replacement sensor models that correspond 
to overlapping images and could emulate the standard triangulation process that refines sensor support data 
parameters. Of course, since the adjustment vector A has no physical relevance, quality assurance checks are 
significantly better when using the standard triangulation. 

6. CONCLUSION 

This paper has presented the background to the use of rational functions and polynomial functions (especially the 
USM) and has determined the use of replacement sensor model techniques which can be used for a wide range of 
imagery. The arguments for and against the use of polynomial functions have been presented and analysed. The 
results from a number of organisations which have tested these have been presented and discussed. 

A critical issue is the lack of complete and rigorous error propagation information. A new method was introduced 
that could alleviate that situation for two of the techniques (RF and USM). It adds increased complexity to the 
techniques, but improved geopositioning performance when using these replacements sensor models may make it well 
worth while. 

The test results reported in this paper show that in the cases reported polynomial functions work well and can be used 
without loss of accuracy compared to rigorous sensor models. However the tests do not report on failures nor on the 
stability of the solutions. In this sense they may be regarded as being limited and further investigation, or some time in 
production use of RFs, is needed. The test carried out with the provision of rigorous error propagation information 
were successful and so a number of the objections to the method are removed. It seems therefore that the use of 
polynomial functions are quite suited to use with many sensors and provide an efficient and accurate way of using the 
data. In addition they provide a means of defining a universal standard for transferring sensor data. However it is 
important that the parameters of rigorous models are transferred wherever possible and that rigour is not sacrificed for 
the sake of expediency. 

We conclude therefore that the USM, and rational functions in general if used with care, are appropriate for use with a 
wide range of sensors. There is no compelling argument that they are suitable for frame cameras, which have none of 
the problems associated with push broom or Radar sensors. Rigorous models for frame cameras should therefore 
continue to be used and work should continue to define a standard for transfer of rigorous sensor models. 
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